All of the material in this patent application is subject to copyright protection under the copyright laws of the United States and of other countries. As of the first effective filing date of the present application, this material is protected as unpublished material.
However, permission to copy this material is hereby granted to the extent that the copyright owner has no objection to the facsimile reproduction by any one of the patent documentation or patent disclosure, as it appears in the United States Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
1. Field of the Invention
The present invention relates to memory management and display control techniques typically used in digital appliances such as Personal Digital Assistants (PDAs), Digital Still Cameras (DSCs), Personal Computers (PCs) and game consoles.
2. Description of Related Art
Electronic systems that display image data often contain a display that allows the user to view portions of a larger image object, and to scroll the viewing window to allow the user to view different portions of that object. Examples of such electronic systems include:
personal digital assistants, which because of their small screen, often display only a very small part of a whole image, such as a map;
digital still cameras, which may include a display using either an integrated display or attached monitor, that allow scrolling through a panorama picture or viewing of a photograph in zoom mode;
game consoles, where games often use a 2D textured background that scrolls as the user interacts with the game; and
personal computers, where an xe2x80x9cextended desktopxe2x80x9d extends beyond the limits imposed by the physical screen size.
Electronic devices with scrolling image displays may integrate display features into a single integrated circuit. These integrated circuits are sometimes referred to as a system-on-chip (SoC). The SoC typically interface to the following additional elements:
a display device which receives a video signal,
a combination of non-volatile memory (e.g. flash) and system memory (e.g. DRAM),
and a number of input and output facilities in the appliance, such as buttons and step motors.
Internally, an SoC may include:
a CPU, which runs the software of the embedded application,
a display DMA controller which reads, directly from memory, data defining pixels to be displayed and sending that data to a display processor which processes that data into a suitable video signal,
an optional xe2x80x9cblock movexe2x80x9d (a.k.a. 2D DMA) accelerator which accelerates the copying of rectangular areas from a source location in memory to a destination location in memory (these operations can be done in software at the cost of reduced performance),
an I/O controller which interfaces with input and output devices,
a memory controller which interfaces with external memory,
a memory arbiter which arbitrates access to the memory between the various processes operating on the chip,
other hardware acceleration blocks, such as a JPEG codec,
and an xe2x80x9con chip busxe2x80x9d interconnecting all of the above.
In the operation of an electronic device with a scrolling display, the image to be displayed is either computed by the CPU or other dedicated hardware block included in the device, or it may be read directly from some other storage device, such as a flash memory. Once the image to be displayed is determined, the image is stored in system memory.
In the example of a digital still camera, the image is usually compressed and is typically read from flash memory, decompressed by the CPU or dedicated hardware, and stored in system memory. This stored image data in this example is then retrieved by a display Direct Memory Access (DMA) controller and is provided to a display controller. The display controller processes and formats the image data as required prior to output to the display device, such as a LCD or a TV.
The DMA controller in this example generates requests to the system memory arbiter to read data that defines the displayed image pixels. The arbiter grants the requests based on considerations such as memory availability and the relative priority of pending requests. When the DMA request is granted, the display DMA controller communicates pixel addresses to the memory controller, which generates the proper control signals to read the pixel data from system memory. Pixel data is usually retrieved in bursts of several pixels at a time in order to optimize memory bandwidth usage. The display controller typically stores the burst of retrieved data in a First In, First Out (FIFO) storage buffer for processing. The display controller then configures the DMA controller to read a new burst of data prior to exhausting the data within the FIFO.
Systems that have scrolling image displays that display a subset of a larger 2D graphics image generally utilize one of two techniques to buffer the image during scrolling.
A first technique, denoted herein as the xe2x80x9csingle-bufferxe2x80x9d technique, is generally used in applications such as extended computer desktops. In the Single Buffer technique, the entire 2D graphics object is mapped into a contiguous segment of system memory. Scrolling is realized simply by changing the base address of the display buffer. The main drawback of this technique is that the size of the 2D graphics object is limited by the amount of system memory available to store the image.
A control program associated with the single-buffer technique first stores the entire 2D object in system memory. A control loop then starts which consists in sampling the user input and updating the display base address to implement scrolling. This simple control program is often merged into a more complex application specific program, e.g. there might be parallel processes that update the content of the 2D object. For example, in the xe2x80x9cextended desktopxe2x80x9d application, when the mouse pointer reaches the edge of the screen, the desktop scrolls to reveal an off-screen part of the desktop. Transfers of data into the buffer of a single image data buffer implementation are not required as a result of scrolling since the entire image is stored in the single data buffer.
A second technique, typically used with digital appliances or 2D game consoles, is referred to herein as the double-buffer technique. The double-buffer technique uses two buffers that are each the size required to store a frame of the image data that is displayed to the user. The entire 2D graphics object is not stored in system memory, only the portions of the image that is or is to be next displayed are stored in the buffer. One buffer is used as a display buffer while the second is used as the update buffer. The next scene is built in the update buffer while the display controller reads data from the display buffer. When the new scene is complete in the update buffer, the functions of the buffers are swapped; the display buffer becomes the update buffer and vice-versa. Simply toggling a data pointer between the two base addresses may be used to rapidly accomplish this switch.
The double-buffer method has several drawbacks. Some of these drawbacks are:
successive scrolling scenes show largely overlapping portions of the 2D graphics object, therefore most of the same pixels are present in both buffers. This duplication of image data results in sub-optimal memory usage;
a given pixel will be written repeatedly into the buffers, at different locations, as long as it is present in the displayed scene. This repeated writing of data into the buffers results in memory bandwidth waste and its corollaries: power waste and system performance degradation;
before the new scene can be built, user input regarding scrolling direction must be known, which can result in slow response time.
A simpler version of this technique uses just one buffer. The new scene in this simpler version is constructed in the same buffer as is used for display. Apart from the smaller memory footprint, it retains all the drawbacks of the double-buffering technique, while adding the drawback of a less elegant user interface. If the update process takes more time than display vertical refresh period, the user of a device with this simpler version will see artifacts, such as image tearing, on the display during an update because the new scene is being written over the previous scene that is in the same display buffer.
The control program that implements the double-buffering technique first builds the initial scene in one of the buffers, buffer A for example. Buffer A is then used as the display buffer. A control loop then begins that samples user input and based on user input concerning scrolling direction, the next scene is built in another buffer, e.g. buffer B. While the new scene is being built, which can take some time, user inputs must be ignored or queued, in both cases the user does not perceive any response to her inputs. When the new scene is ready, the functions of the buffers are swapped, buffer B becomes the display buffer and buffer A becomes the update buffer.
Therefore a need exists for a technique that circumvents all the above-described drawbacks by providing simultaneously:
low power operation by writing a given pixel only once to system memory, instead of many times,
low memory footprint, by avoiding the storage of the entire 2D graphics object in system memory and the duplication of pixels in memory
good system performance by minimizing memory bandwidth usage
good response time by anticipating user input
The present invention provides a system and method for buffering and accessing image data. The present invention stores image data in a buffer that acts as a display buffer and that is larger than the data that is displayed at a given time. The buffer therefore retains a rectangular portion of the 2D graphics object that is larger than the rectangular portion currently being displayed. The present invention also provides a novel and efficient method and apparatus to store and retrieve the data within the buffer.